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Abstract 



This review assembles what is known from studies and analyses of large-scale systemic reform 
initiatives aimed at mathematics and science education, especially those undertaken by state 
governments and the National Science Foundation. The review concentrates on qualitative 
investigations, which reveal whether and how these initiatives converge on the improvement of 
instruction. The evidence to date, though inherently incomplete, has much to say about the logic 
of systemic reforms, the avenues by which these reforms may reach the classroom, and the kinds 
of classroom-level impacts that are apparent so far, or are likely to occur in the near term. The 
implementation and effects of these reforms can be understood productively in the theoretical 
terms applied to the study of innovation and change, implementation of governmental policies, 
and professional and organizational learning. 
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Introduction 



It's a long way from the framing of a reform policy at the level of the system to the mathematics 
or science lessons taking place in elementary or secondary school classrooms. That distance begs 
many questions about how and whether the class of educational improvement strategies known 
as "systemic reforms" are traversing the levels and sectors of the system in such a way that 
teaching and learning are enhanced. In brief: How, and in what forms, are the current crop of 
systemic reform policies reaching the mathematics or science classroom? What kinds of impacts 
on classroom practice have they had to date? 

This review assembles what is known in answer to these questions from a variety of studies and 
analyses of large-scale systemic reform initiatives, especially those undertaken by the National 
Science Foundation and various states. The review concentrates on evidence emerging from case 
studies and qualitative examinations, on the premise that this kind of research is best able to 
reveal precisely how these strategies do or don't converge on the improvement of instruction. 

The review is being done at a time when few systemic reforms of any scale, with the possible 
exception of several state initiatives, have been in place for many years. Studies to date report on 
the progress of these reforms over the first half dozen years at most, and often over only the first 
year or two of a systemic reform process. Thus, the evidence so far is partial and does not fully 
capture what these ambitious strategies may accomplish over the long haul and how they can do 
it. Nonetheless, the evidence has much to say about the logic of systemic reforms, predictable 
paths and diversions in their implementation, and the current or likely effects at the classroom 
level. 



Evidence Regarding the Implementation and Effects of Systemic Reforms 

In assembling evidence about the implementation and effects of systemic reforms, particular 
attention was paid to (1) the design and execution of these reforms at the policymaking level; (2) 
the extent of reform impact on individual classrooms, and manner in which this is taking place; 
(3) the avenues by which reforms are reaching, and exerting influence on, classrooms; and (4) 
the contexts that are mediating the influence of systemic reforms. 

By combining emerging findings regarding the National Science Foundation initiatives (in their 
first few years) with what is known about the longer term reform ventures in California, 
supplemented by emerging evidence from other states’ initiatives (e.g., Michigan, South 
Carolina, Kentucky, and Vermont), one can construct a rough, if incomplete, empirical picture of 
systemic reforms in action, especially those emanating from state sources. 

Theme and Variations in the Design and Execution of Systemic Reform Strategies at the 

Policymaking Level 

While there is considerable variation in the precise mix and orchestration of components, all of 
the systemic reform strategies in mathematics and science education are multifaceted courses of 
action combining the setting of curricular standards, the development of ambitious frameworks 
(and often materials to accompany them), the alignment of assessments in part or completely 



1 




9 



with these frameworks, and the allocation of resources for various forms of professional 
development in support of the framework vision. The reform strategies chiefly vary in terms of 
how strong a role is intended for centrally determined standards, how much attention is paid to 
building broad community and professional support for the reforms, and how much the initial 
preparation and licensing of educators is included. The reform strategies are evolving — most 
clearly seen in the longest running instances of systemic reform policy (e.g., in California) — as 
new features of the reform strategy (e.g., curricular "replacement” units, offered as alternatives to 
commercial textbooks) are grafted onto the original set of components to remedy apparent 
deficiencies (e.g., the poor quality of most textbooks on the market). 

How, and How Extensively, Systemic Reforms Appear to Have Reached the Target of Policy 
(T eachers, Teaching, Learning, Classrooms) 

Within various classrooms, many science and mathematics teachers have been touched by the 
reforms, and in a variety of ways. Teachers’ early responses appear decidedly mixed: there are 
signs of attempts to realize some or many aspects of what the reforms advocate in classroom 
practice, although little evidence of teachers fully grasping and internalizing the reform vision. In 
general, the more easily imported practices (e.g., the use of manipulatives in elementary grades 
mathematics teaching) have become part of teachers’ repertoires, while the full understanding of 
what these practices may mean has not. The implementation of reforms in the classroom is often 
piecemeal, involving discrete elements of what is called for (e.g., segments of instruction related 
to different sciences) but not a coherent whole (e.g., a fully “integrated” science curriculum 
across the year). Understandably, teachers rely heavily on what they already know best. Thus, 
classroom “use” of reform-advocated practices may support or subvert reform intentions, and 
teachers are often unaware of how much or how fundamentally their practice is changing. 

Longer term, gradual changes are beginning to be documented in some classrooms exposed to 
systemic reforms over time, while other classrooms reveal little change. Reported cases to date 
suggest that teachers engage in different areas of learning and end up with different pictures of 
what the reforms are all about. There is clear evidence of teachers engaging in significant new 
learning about their practice. 
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The Avenues by Which Systemic Reforms Are Exerting Influence on Classrooms 



Systemic reformers have tried to reach the classroom through multiple avenues, but appear to 
have invested the greatest energies to date in influencing (1) the interrelationship among 
requirements related to curriculum and assessment, (2) the environment of ideas to which 
teachers might be exposed through professional networks and other forms of professional 
development, and (3) the mobilization of state and local coalitions of actors in support of reform 
goals. Less energy has been expended conceiving, developing, or maintaining the many forms of 
support that teachers might require in order to carry out the reforms in question. Relatively little 
attention has been paid to altering the supply or quality of the teaching force. 



Contexts That Are Mediating the Influence of Systemic Reforms on Teaching and Learning. 

There is considerable evidence accumulating about the ways reform messages are reshaped by 
various contexts in which teachers work. Local contexts vary considerably, and the clearest 
contrasts appear between high- and low-capacity contexts. Schools with cohesive staffs who are 
used to the demands of ambitious reforms (as in schoolwide restructuring attempts) and who 
have developed collegial learning communities within and across schools appear the most 
receptive to what the new reforms are calling for. 

Correspondingly, at the district level, those in a position to relay and reinforce the message 
emanating from the reform policy sources have varying capacities and willingness to perform 
this intermediary role. There is case evidence of district-level staff engaging in extensive 
attempts to understand reform visions in some depth and, based on that understanding, to provide 
the most helpful forms of support to teachers. In more “ordinary” contexts, district-level actors 
appear to be implementing reforms without grasping deeply what these are all about and 
transmitting to teachers guidance that is viewed as unhelpful, intrusive, or both. As in schools 
and classrooms, district-level people implement the reforms within the realm of what they know 
and understand of the subject matter. Furthermore, districts maintain commitments to a variety of 
other policies, not all of which are compatible with the direction of systemic reforms (e.g., basic 
skills-oriented programs for supposedly disadvantaged youngsters). 

Outside of districts, professional support networks-some purposely created by state government, 
others arising independently-offer another layer of opportunities for teachers and administrators 
to become more knowledgeable about content, techniques, and instructional philosophies. 

Access to these networks is mixed, and participation in them is no guarantee that reform 
messages will be reinforced or amplified. Nonetheless, engagement of teachers in professional 
communities that transcend school or district boundaries appears to be an important feature of 
the high-capacity settings that have done the most to incorporate reforms into practice. 

Interpreting the Implementation and Effects of Systemic Reforms 

The unfolding patterns of systemic reform can be interpreted in more than one way. Three 
conceptual perspectives offer complementary accounts of what is happening and why: 
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• Systemic reform as innovation and change. This perspective construes what is happening as a 
sophisticated planned change strategy to promote the spread and adoption of curricular 
innovations. The strategy encounters predictable resistances in the units within which change 
is supposed to occur. This perspective locates the source of resistance in the frequent lack of 
fit between the nature of the innovation and the adopting units. Over the long term, this 
perspective visualizes an adoption process in which a gradually accumulating critical mass of 
"users" proceeds through predictable "stages of concern" regarding new approaches to 
mathematics or science teaching and eventually accepts these as part of their practice. 

• Systemic reform as implementation of higher level policy. Where the first perspective focuses 
on the transfer of a relatively discrete practice to end-users, a second perspective highlights 
the role of diverse contexts and actors in the intergovernmental chain between policy source 
(e.g., the state or district level) and policy target (the classroom). This framework helps to 
explain why systemic reform policies might stimulate such varied responses at each level of 
the system traversed by reform messages, but leaves somewhat unexamined what the policy 
might require of all who seek to realize it. 

• Systemic reform as professional and organizational learning. A third framework zeroes in on 
the individual and collective struggle to master new ideas at all levels of the system as the 
reform process unfolds. In these terms, the patterns of implementation and effect are largely a 
story of incomplete professional and organizational learning occurring as the learners 
(teachers, administrators, curriculum coordinators, staff developers) encounter the often 
limited "teaching" (by those who make or promulgate reform policies, as well as by the 
policy itself) and learning opportunities (created by, or around, reform policies). The longer 
term prospects for the policy's success depend in large measure on the quality of the reform 
policy's "pedagogy" and learning resources over time. 

Making Sense of the Power and Limits of Systemic Reforms at the Classroom Level 

The available evidence regarding systemic reforms supports various claims regarding the way 

these reforms are making their way to the mathematics or science classroom. These reform 

initiatives — 

• Are hastening the diffusion of attractive , if challenging, ideas about subject matter and 
pedagogy. 

• Are mobilizing commitment, resources, and actors on an unusual, in some instances 
unprecedented, scale. 

• Have made a modest contribution to the rationalization of regulation and resource allocation 
at the "top” of the system. 

• Are reaching teachers in large numbers but are not necessarily transmitting reform messages 
that are faithful to the original intention or fully understood by most teachers. 
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• Have stimulated a great deal of new learning among many individuals throughout the 
system. 

But much of the story of classroom impact has yet to be told. Judgments of the ultimate power 
and limits of systemic reforms to influence classroom practice will depend, in part, on the 
standards of "success" to which the policy is held. If observers expect teachers to embody fully 
the ambitious curricular visions that drive these reform initiatives, then disappointment is almost 
inevitable. If, on the other hand, observers settle for widespread grafting of reform ideas onto 
familiar practices, or a gradual upward shift in "system averages," then systemic reforms may 
- prove themselves to be a potent instrument of instructional improvement. 

The capacity for systemic reforms to support long-term professional learning, appropriate 
adaptations at the local level, and adoption by a critical mass of teachers will also depend, in 
part, on public perceptions and patience. To the extent that the public likes what it hears about 
the process or products of reform, then it will tolerate — or even support with varying degrees of 
enthusiasm — the slow, complex attempt to make the educational system demand and support 
more challenging instruction in mathematics and science. But public patience is in short supply 
these days; systemic reforms have much to do to portray their policy initiatives in a light that will 
engender and sustain public confidence in schools. 
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Framing the Review 



Systemic reform in precollegiate mathematics and science education: its logic is clear and in 
many ways compelling. Its scope is broad and bold. Its rhetoric is attractive and has been readily 
picked up by policymakers at various levels in the nation's educational systems. Its promises are 
sweeping. But can such reform initiatives, conceived at the highest levels of educational systems, 
stimulate new thinking and improve practices in mathematics and science classrooms across the 
land? If so, how has this happened so far and how might this happen in the future? Given what 
we know about professional or organizational learning, the implementation of educational 
policies, and the process of change in organizations in general or schools in particular, it is easy 
to imagine many ways in which the reformers' intentions might not be realized. It is equally easy 
to dismiss a complex and promising policy idea before one fully appreciates its potential. To 
grasp what such reforms may accomplish involves an ongoing dialogue between diverse and 
powerful ideas about policy-to-practice connections and emerging evidence from instances of 
systemic reform. 

In the pages that follow, I carry on that dialogue by drawing together theory and threads of 
evidence regarding the forces, conditions, and processes that intervene between large-scale 
educational reform efforts and practices in mathematics and science classrooms at the elementary 
and secondary school level. In reviewing what is known, I bring theoretically based claims and 
frameworks to bear on the evidence that has emerged so far from the most visible and sustained 
systemic reform efforts in science and mathematics education. I pay special attention to systemic 
reforms supported by the National Science Foundation (NSF), alongside other initiatives that 
pursue similar aims although not necessarily in these two subject areas. 

The sources of my review include (1) education journals that deal with reform, policy, or 
mathematics and science education; (2) a search of the ERIC database; (3) studies of NSF- 
supported systemic reform initiatives (e.g., as reported in interim documents and project progress 
reports); (4) relevant books and book chapters; and (5) fugitive sources such as conference 
reports, association documents, or unpublished conference presentations. Taken together, these 
literatures offer theoretical work, policy advocacy, some wishful thinking, descriptions of 
initiatives in their early stages, and partial evidence about systemic reform dynamics and effects. 
But most instances of reform are too recent and too big to have "run their course" in a way that 
permits confident data-based conclusions about them. Nonetheless, based on what has been 
written to date, there is much to say about the evolving logic of systemic reforms, predictable 
paths and diversions in their implementation, and likely effects at the classroom level. Insights at 
this stage can help to guide how we continue to look at systemic reforms in progress or on the 
drawing board, as well as suggest refinements or adjustments in the reforms themselves, or our 
expectations of them. 

The review concentrates on evidence from case study accounts and other qualitative sources, for 
the following reasons. Although such representations of reform effects are unlikely to capture 
broad tendencies and trends, they are especially likely to reveal whether systemic reform 
activities can reasonably be credited with actual changes in teachers' thinking or instructional 
practice. Second, such accounts are most likely to reveal how systemic reform messages make 



6 



O 

ERIC 



14 



their way into classroom practice, if at all, and how they might be changed in the process. Given 
the complexity and ambiguity of the policy messages and mechanisms in this instance, the ability 
to trace plausible chains of influence takes on special importance. Doing so does not make large 
sample quantitative research inappropriate 

A word or two about vocabulary and boundaries for discussion. I am using the term "systemic 
reform" to refer to a class of policy strategies that go by various other names, including 
"alignment" (Hill, 1995), "coherent policies" (Fuhrman, 1993), and "standards-based reform" 
(Sykes & Plastrik, 1993; McLaughlin & Shepard, 1995). While there are shades of difference 
among these terms and in the thinking of those who use them, I will argue that all rest on a 
common set of beliefs about the potential of policy to influence educational practice in a 
comprehensive way and, therefore, they deserve to be considered together. Systemic reform 
strategies have been most visibly discussed and instituted at the state level, but appear at the 
national, regional, or local levels as well, and I will include all levels as part of the discussion 
that ensues. 

My argument unfolds, first, by reviewing briefly the premises on which systemic reform thinking 
rests and noting the better known manifestations of this thinking in particular initiatives. 
Following that, I offer a framework for examining what is known or claimed regarding systemic 
reforms and then use it to consider evidence regarding current NSF-supported initiatives and 
others (e.g., California's curricular reforms of the past decade). Stepping back from the evidence, 

I then interpret it from three vantage points, as an instance of micro-level innovation and change, 
as a manifestation of macro implementation dynamics, and as a case of professional and 
organizational learning. The paper concludes with reflections on what we do and don't know 
about systemic reform and what we might examine further to understand the accomplishments 
and potential of these reforms in greater depth. 

What Systemic Reforms in Mathematics and Science Education Assume 

Approaching the problem of improving mathematics and science education through systemic 
strategies is a logical outgrowth of the less comprehensive, although still ambitious, reform 
attempts of the last four decades. Such earlier reforms targeted various facets of the educational 
system: curriculum (as in the NSF-supported curriculum improvement investments of the 1960s 
and 1970s), instruction (as in the movement supporting basic skills instruction in the latter part of 
the 1970s and early 1980s), teacher preparation and professional development (as in the federally 
funded Teacher Corps or NSF-supported summer institutes for mathematics and science teachers 
in the 1960s and 1970s), assessment (e.g., the movement to develop criterion -referenced tests as 
part of measurement-driven instructional improvement strategies), and graduation requirements 
(e.g., as in the wave of state actions in the early 1980s to increase the number of mathematics 
and science courses students needed for a high school diploma). Add to these, initiatives aimed 
at features of the school as a whole such as school governance (e.g., through the wave of 
attempts to instill school-based management) or school climate and ethos (as in the Effective 
Schools movement of the past several decades). But prior to the 1980s few attempts had been 
made to address all or most of these elements at once. 
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By investing repeatedly in these discrete aspects of the system and noticing that initiatives with a 
limited focus brought mixed results — that is, were always constrained by other aspects of the 
system that were not the focus of concern — policymakers set the stage for policy designs that 
addressed the most central elements of the system at the same time. By the late 1980s, building 
on earlier work (e.g., Floden et al., 1988), systemic reform thinking had been articulated clearly 
in widely disseminated pieces by Smith and O'Day (1990) and others (e.g., National Governors' 
Association, 1989). Soon thereafter, a number of systemic reform initiatives were underway or in 
the design stage, and the rhetoric of the new class of reforms started to take concrete form. 

In the mathematics and science arena, the earliest of these initiatives was set in motion at the 
state level, most visibly in California, which commenced in the mid-1980s to reform 
mathematics education, and subsequently science teaching (along with other curricular areas), 
through systemic alignment of curriculum frameworks, textbook selection, and assessment. 
Shortly thereafter, departing substantially from its conventional pattern of small grants to 
individual principal investigators, the NSF started contributing to state-level reforms of this sort 
through large-scale grants to state coalitions proposing to revamp science and mathematics 
teaching by various systemic strategies (National Science Foundation, 1994). To date, 26 states 
have received these grants to support attempts to align teacher preparation, curriculum, materials 
selection, the development of assessment and accountability systems, and public perceptions of 
science and science education. 

Other policymakers and reform groups picked up the hue and cry, and by the early 1990s 
systemic reform strategies of one kind or another were being supported in school districts and 
regions, as well as nationally through the efforts of large private foundations in addition to 
federal agencies and Foundations. For example, following a logic similar to that of its Statewide 
Systemic Initiative (SSI), NSF added to its portfolio large systemic grants to regional consortia 
serving rural areas (through the Foundation's Rural Systemic Initiative) and to large urban 
districts (through the Urban Systemic Initiative). Drawing on various sources for support, other 
localities have embarked on their own initiatives aimed at various elements of the district 
"system" that might enhance or constrain the teaching of science or mathematics. 

While they differ in the scale and level of the "system" or the systemic elements they include, 
these initiatives bear a strong family resemblance to one another. In essence, they share the 
following premises: 

A major constraint on the quality of science or mathematics teaching lies in the lack of alignment 
among key elements of the system , either because elements directly contradict one another (as 
when tests oriented towards discrete basic skills are retained while curriculum emphasizes 
advanced skills) or simply ignore one another (as in textbook choices made without reference to 
teachers' preferences, beliefs, or knowledge base). Either way, the policy environment sends 
mixed messages to practitioners, and so, the argument goes, classroom teachers are likely to miss 
opportunities to provide students with a more coherent instructional experience and, if they pay 
close attention to the mixed messages, become confused or resentful, as well. 

Better teaching of mathematics and science will result when all elements of the system that bear 
most directly on the classroom — especially those dealing with what is taught , how it is taught , 
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how learning is assessed [ how teachers are prepared and supported , and how they are held to 
account for student performance — are aligned with challenging standards embedded in a 
coherent, compelling reform vision that reflects professional consensus among scientists and 
educators. Embedded in this premise are two ideas: First, the standards to which mathematics 
and science teachers hold themselves and their students should be higher and more explicitly 
stated; second, teachers are most likely to bring their practice in line with these standards, and 
thereby improve their practice, if their training, the selection of materials and texts, content of 
tests, and so on are all "on the same page." 

The lack of alignment is best addressed at its source — that is, at the level at which policies and 
structures guiding each systemic element are set. Thus, for example, it is assumed that in many 
states curriculum frameworks, requirements for teacher certification, testing systems, and the like 
originate in the minds and actions of state-level actors, who establish these elements of the 
system with reference to professional and local constituencies but also, in principle, in 
relationship to one another. Greater alignment is most likely, the argument goes, if these 
elements of the system are consciously and explicitly set up to reinforce one another by the 
groups or individuals in charge of them. To be sure, levels of policymaking may vary, as in states 
with a strong local control tradition or in districts that undertake a "systemic" reform initiative 
(even though not all the above elements of the system reside there). In such instances, alignment 
is assumed to occur lower down in the system. 

Though driven by high-level policy action, systemic reform strategies are not incompatible with 
efforts to enhance local discretion and professionalism; in fact, some formulations of systemic 
strategies treat governance reforms aimed at maximizing school discretion as indispensable 
aspects of a fully developed systemic reform strategy (e.g., Smith & O’Day 1990; National 
Governors' Association, 1989; Fuhrman, 1993). In essence, the policy approach assumes that, 
given some broad consensus on overall goals of education embedded in the standards toward 
which the system is aligned and a sensitive means by which to assess whether those goals are 
reached, educators can be given a great deal of discretion in the way they organize schools, 
design curriculum, and approach instruction. 

While there is widespread acceptance by many policymakers and observers, these assumptions 
are not without criticism. The principal concerns have to do with standardization and 
centralization, local initiative and professionalism, equity, political dynamics, and institutional 
cultures. First, observers worry about the emphasis placed on standardization and centralization 
by most current systemic strategies (e.g., Clune, 1993). These features of the reform strategy are 
either unnecessary, counterproductive, or simply incompatible with the decentralized governance 
arrangements that pertain in the United States. Second, given the hierarchical, rule-based nature 
of public educational systems, some observers argue that systemic reforms will inevitably 
generate compliance pressures on actors at the local level, especially teachers, who will still see 
themselves as dominated by mandates from above (Hill, 1995; Scheurich & Fuller, 1995). The 
teachers' natural response, it is argued, will be to avoid taking initiative and concentrate on ways 
to protect themselves in the face of bureaucratic scrutiny. Third, in a system tied to higher 
standards, there is also the worry that systemic reform policies will unfairly and negatively affect 
the educational prospects of certain groups of students, those whose opportunities for learning do 
not enable them to master what will be tested on standards-based assessments (Hill, 1995). 
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Fourth, many are unconvinced that systemic strategies can weather the political dynamics of 
reform in the pluralistic context of schooling. Not only is consensus on goals likely to be difficult 
throughout the system, it may only be achieved through agreements that bundle together 
contradictions and compromises (Scheurich & Fuller, 1995) or through vehicles such as the 
recently issued draft of the National Research Council’s National Science Education Standards 
(1996) which some observers see as vague enough to gain support from a wide range of 
viewpoints but not specific enough to guide practice in any meaningful way (Donmoyer, 1995).* 
Furthermore, critics argue that whatever coherence is achieved in policy design will inevitably 
succumb to the political pressures that produced the fragmentation of the system in the first place 
(Brandon, 1993; Hill, 1995). Finally, some critics see systemic reforms as having limited 
capacity to influence teacher beliefs and enduring traditions of classroom organization, the things 
that make the most difference in school and classroom culture (Cuban, 1995; Page, 1995). 

These criticisms are more conceptual than empirical at the moment, for there has not yet 
developed a sufficient base of evidence to test these explanations fully. In addition, the nature of 
the phenomenon being talked about is complex enough to defy easy investigation. But however 
slim the evidence base, the critics' concerns highlight particular aspects of the system that bear 
more careful scrutiny, both in a conceptual sense and in actual practice. Before directing 
attention to them, however, we need to agree on a set of framing ideas with which to analyze 
what has happened or might happen in response to a systemic initiative. 

A Framework for Examining the Influence of Systemic Reform on Classroom Practice 

At the heart of systemic reform thinking is a hypothesis about the influences that the act of 
systemic alignment might have on teachers and learners in classrooms. To frame the way one 
considers these influences, it is helpful to be explicit about four conceptual elements: (1) the 
target of policy (teaching and learning of mathematics and science in classrooms); (2) the 
instrument of reform (what counts as the mathematics and science reform "policy”); (3) the 
avenues of influence (the routes by which reform policy could conceivably affect classroom 
practice); and (4) the contexts and conditions that are most likely to mediate policy influences. 

As I have argued at greater length elsewhere (Knapp, 1995), rich conceptions of each element 
are necessary to understanding policy effects on the classroom. 

Conceptions of Teaching and Learning 

Embedded in policies themselves, and in the minds of those who make, observe, or respond to 
policies, are conceptions of teaching and learning — that is, ideas about what is taking place in 



* Widely promoted curricular standards aren't always viewed as diffuse. An equally 
common critique — for example, regarding American Association for the Advancement of 
Science’s (AAAS) Benchmarks for Science Literacy (1993) — is that the standards are too 
ambitious and specific. Moreover, the National Council of Teachers of Mathematics (NCTM) 
Standards (1989) represent a substantial degree of consensus in the more homogeneous 
mathematics education community. 

classrooms that lie within the province of policy. Not all aspects of the buzzing confusion of a 
classroom can be reflected in the policy designs, but a short list of features that might be 
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reachable by policy clearly helps to characterize both the target of policy and the range of its 
possible effects. Consider this as a starting list: 

• The conceptions of scientific or mathematical "knowledge" held by teachers, learners, and 

others. , 

• The teachers' beliefs about good pedagogy, the learners they are teaching, and learning itself 
(including their own). 

• The teachers’ mastery of content and "pedagogical content knowledge." 

• The teachers' repertoires of subject-related practices and strategies for coping with classroom 
contingencies. 

• Teachers' decisions about what content to teach and how to engage learners in the learning 
process, including decisions about how to group or organize the learners. 

• The actual structure and demands of academic tasks in which learners engage. 

This list of elements leaves out a host of factors that teachers and learners experience such as the 
"interactive, often tacit, sociocultural processes" (Page, 1995) at work in classroom life. But the 
elements arguably include central features of the teaching and learning enterprise that have a 
demonstrable relationship to both learning opportunities and actual learning itself. To be sure, 
some elements on the list are more susceptible to influence than others. For example, it may be 
simpler (although not simple) to alter teachers' decisions about what mathematics to teach in 
seventh grade than their beliefs about good pedagogy. 

Conceptions of Reform "Policy” 

A rich conception of the target of policy begs an equally rich conception of the instrument 
designed to affect classroom life. While it is tempting to treat "policy" as a formal statement of 
intend ons-for example, a page in a district Policy Handbook or the document setting forth the 
state's expectations for content and pedagogy in a subject area-doing so does not take into 
account the enduring tension between intention and action that marks all policymaking (Elmore 
& Sykes, 1992). One broader view conceives of policy as a "purposeful course of action by 
individuals at higher levels in the system designed to guide, direct, or support actions at lower 
levels of the system across settings and across time" (Knapp, 1995, p. 2). This kind of conception 
includes both intentions and actions within its scope, both what policymakers proclaim and what 
they (and others who work with them) do to bring those expectations about. Obviously, such a 
view subsumes some of what is conventionally referred to as "policy implementation," at least, 
what is done by higher-level actors to implement a policy. 

Such a course of action may include various elements, among them sending broad signals, 
setting (or waiving) specific requirements or regulations, allocating resources of various kinds, 
establishing and levying sanctions for failure to comply with requirements, rendering assistance, 
setting up channels and expectations for the flow of information, and (re)allocating authority. All 
of these, or judicious combinations of some of them, are bound together into a strategy aimed at 
guiding (or controlling), motivating, and supporting the efforts of groups and individuals 
throughout the system. 
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This conception leaves unresolved a number of matters, such as precisely where to draw the 
boundaries of the "system" included in the “systemic” reform of mathematics and science 
education or how much the school as a whole is implicated in the reform of particular subject 
areas such as these. But, in principle, a broad conception of policy of this sort expands the realm 
of actions and actors at the policymaking level that may exert an influence over classroom-level 
activities. 

Avenues by Which Policy Might Influence Practice 

Having the target of policy and the instruments of policy clearly in view leads one to the 
question of how the two might be connected-in short, the nature of the influence the one exerts 
on the other and the routes by which influence is exerted. While there is good reason to believe 
that each exerts influence on the other (see Cohen & Ball, 1990), I will concentrate here on one 
* direction of influence, that of policy on practice. Consider the following avenues of influence: 

• Influencing the environment of ideas about science , mathematics, pedagogy, and learnes. 
Here, policy can introduce, popularize, and validate ideas that shape professional 
conversations, public perceptions, and the attention paid to particular facets of the 
mathematics and science teaching enterprise. 

• Influencing coalitions of actors who press for (or resist) improvement in practice . Here, 
policy can legitimize and empower new configurations of players at one level or another, 
including coalitions of teachers whose philosophies and approaches are compatible with the 
direction of policy. 

• Influencing the environment of requirements that shape and constrain practice. Here, policy 
can add to or subtract from the total array of requirements and may also alter the relationship 
among them (e.g., linking one set of requirements to another). Of particular relevance to 
teachers are requirements related to what is taught, how it can be taught, and how it is 
assessed. 

• Influencing the forms and availability of support for instruction and instructional 
experimentation. Here, policy can provide direct support through resource allocation, the 
creation and dissemination of appropriate materials and equipment (especially relevant to 
science teaching that emphasizes hands-on experience), and also the creation of professional 
development opportunities and the provision of other forms of technical assistance. 

Indirectly, systemic reforms can support instruction by stimulating various kinds of local 
support arrangements. 

• Influencing the supply and quality of teachers. Here policy can alter the kind or number of 
newly trained teachers, or the nature of credentialing. 

While not exhaustive, these five avenues begin to capture the multiple routes available to policy 
to reach the classroom, by tracing some obvious ways that ideas, people, political pressures, 
resources, and constraints come to bear on classroom action. Some routes are more direct than 
others, and I do not assume that policymakers are fully aware of them, nor that they intentionally 
design their policies to exploit all of these avenues simultaneously (to do so would imply that 
policymaking groups possess an almost superhuman grasp of the system as a whole). 
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Contexts That Mediate Policy Influence 

Work on the interaction between teaching and its multiple contexts (Floden et ah, 1988; 
Rosenholtz, 1989; Little & McLaughlin, 1993; Cohen, McLaughlin, & Talbert, 1993; 

McLaughlin & Talbert, 1993) helps to frame a final conceptual element. Simply put, this line of 
thinking and research asserts that teaching (the target of policy) can only be understood within 
various nested contexts. Working outward from those that are most immediate in the working 
lives of teachers, these contexts include the classroom context (which presents teachers with a 
daily set of interactive personal demands), the social organization and culture of the school 
(which enshrines enduring traditions of the way learners and teachers relate to one another and 
for what purposes), the family and community context (which conditions what learners expect of 
themselves and what educators might expect from them), the district and state policy 
environment (which constrains and supports what school level actors do), and the environment of 
professional relationships (which nurture educators' sense of identity, contact with knowledge 
bases, and access to professional advice and counsel). 

Just as teaching itself is beholden to these various contexts, so is policy, which reaches teaching 
through these contexts. Generally speaking, the contexts that are more remote from the 
classroom are more directly reachable by higher-level policy action, and vice versa. Nonetheless, 
in principle, the avenues of influence described above use all five contexts in some degree as the 
medium of communication between policy source and the classroom. 



The Evidence Connecting Systemic Reforms with Change in Classroom Practice 

The framework just described provides us with a tool to systematically assemble and sift what is 
known about the implementation and effects of systemic reforms. The empirical evidence 
regarding what happens when systemic reforms are initiated comes from various sources, the 
most useful of which are large-scale studies that track reform effects over time. To date, the 
evidence is far less complete than we might like, although that is not surprising given the breadth 
of the phenomenon we are trying to understand and the recency with which many systemic 
reforms have been instituted. I review below evidence regarding NSF's systemic reform 
investments, other systemic approaches to mathematics and science education improvement, and 
broader reform efforts that aim at more than a single subject area. I concentrate on evidence from 
qualitative sources, on the premise that this mode of inquiry is most likely to uncover the manner 
in which systemic reforms take advantage of particular avenues of influence. 

Emerging Evidence Regarding NSF's Reform Initiatives 

So far, the principal lines of evidence regarding any of NSF's reform initiatives come from a 
large-scale national evaluation of the SSIs (Consortium for Policy Research in Education 
(CPRE), 1995; Shields, Corcoran, & Zucker, 1994; Zucker, Shields, Adelman, & Powell, 1995; 
Zucker & Shields, 1995), supplemented by several attempts to assemble "lessons learned" 
regarding systemic reform efforts at the state level (Horizon Research, Inverness Research 
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Associates, & Westat, 1994) and in urban school districts (St. John, Century, Tibbitts, & Heenin, 
1994). The main themes in these sources are summarized below. 

NSF's Systemic Reforms at the State Level 

The returns from the first and second years of the Foundation’s initiative, as represented by the 
National Study of the SSIs and bolstered by insights from a Wingspread Conference of SSI 
participants part way through their five-year grants, yield a picture of this class of reforms in the 
early stages of enactment and implementation. These information sources tell us most about what 
the state-level actors and others allied with them have been doing to generate the visions that will 
guide systemic reform, how they tried to forge and sustain coalitions of diverse interests in 
support of these goals, how they have developed strategies that will realize these visions, and 
how they have mobilized resources to realize these strategies. 

To date, participants in many NSF-supported statewide initiatives have been understandably 
preoccupied with vision-building and the search for consensus among diverse groups that have 
become part of the SSI process within each state. SSI leaders, their staffs, and the coalitions they 
work with have also been immersed in the fashioning and refining of strategies for realizing the 
vision. With much variation across states, each has created (or is still evolving) a strategy with a 
unique combination of the following elements: support for professional development of teachers, 
infrastructure development, adjustment of state policies to bring them into greater alignment, 
development and dissemination of new content and materials, funding for local systemic efforts, 
support for model schools, attempts to reform teacher education, and campaigns to mobilize 
professional and public opinion in support of reform goals (Shields et al., 1994). Participants 
distinguish the ’’technical” aspects of developing consensus around standards and strategies from 
the ''political” aspects and have been surprised to find the political work so difficult and 
demanding. 

Because of their scale, developmental nature, and origin in state-level (or cross-district) 
coalitions, it is not surprising that, as a group, these initiatives in their second year of operation 
exhibit connections between reform and classrooms that are uneven, often nonexistent, or simply 
hard to trace. Most of what we know about these connections comes from case studies of SSI 
reforms in three states — Montana, Delaware, and Connecticut (Zucker & Shields, 1995). 

Montana, for example, has mounted one of the more focused initiatives, which concentrates on 
high school mathematics, although it has pursued work at other levels and in science as well 
(Zucker & Hawkins, 1995). In its first two years, the reform produced a detailed vision of 
mathematics curriculum for the high school featuring integration of topics, connections with 
other disciplines, and heavy use of technology (e.g., graphing calculators, computers). 

Curriculum materials were produced by SSI-sponsored developmental teams for the ninth and 
tenth grades and piloted in selected schools; the freshman-level materials, which were developed 
first, have been (voluntarily) introduced into a fifth of the state's high school classrooms. Half of 
the state’s mathematics teachers have been involved in professional development sessions of 
various kinds designed to introduce them to the new curriculum, and these sessions have been 
much more intensive (e.g., six-week summer institutes) than more typical patterns of staff 
development elsewhere (see Little, 1989; Moore & Hyde, 1983). Finally, grants from the 
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Legislature have enabled most high schools in the state to get complete sets of requisite 
technology such as graphing calculators. 

The reports to date leave lots of uncertainty about what precisely Montana's high school teachers 
are doing with these materials. Many are using the materials to structure much of their current 
teaching, and enthusiasm is reportedly high, although there are clearly signs of struggle (e.g., 
how exactly should I be using cooperative learning groups in my classroom?), uneven adoption 
(e.g., in some classrooms but not others within a school), misinterpretation of its intent (e.g., by 
tracking only the low-performing students into the new mathematics curriculum), and even overt 
resistance (e.g., the new curriculum just doesn't fit my style; Zucker & Hawkins, 1995). 
Nonetheless, considering the adopting classrooms in contrast with nonadopting ones, there is 
evidence that students who experience the new curriculum are progressing as well as or better 
than comparable counterparts without this exposure. 

Montana's forward momentum and numerous signs of connection between this reform and a 
large proportion of relevant teachers and classrooms within the state contrasts sharply with the 
SSI in Delaware. In the latter state, the first two years have largely been spent building 
partnerships among relevant groups, developing statewide standards for mathematics and science 
education, instituting an interim assessment instrument with some performance-based measures 
that capture learning related to the new standards, and nurturing 17 demonstration sites charged 
with the development and display of teaching strategies linked to the new standards (Adelman, 
1995). Unlike Montana, in which there is widespread awareness and name recognition of the 
new mathematics reform, the reform initiative in Delaware has not captured the attention of a 
large number of the state's teachers. Most of the action has taken place at the demonstration 
school sites, which have been understandably at differing stages of sophistication about reform 
and their own restructuring process. Some of these schools had extensive prior experience with 
reforms and had developed a relatively cohesive sense of collective purpose by the time the new 
mathematics and science initiative appeared on the horizon. In others, the teams of teachers 
associated with the SSI appeared to operate as "isolated pockets of reform activity" (p. 67). 

While teams in these schools have been busy preparing and using high-quality demonstration 
units for external use, little effort has yet been devoted to building and serving networks of 
schools and teachers elsewhere. Other forms of SSI outreach to the state's teachers as a whole 
have been undertaken — for example, a one-day professional development event to explain the 
new standards to all teachers in the state — but these efforts are a far cry from the kinds of 
intensive forms of professional exposure that are generally agreed as necessary to introduce fully 
new curriculum. 

Acknowledging the early stage in their development, it is still useful to review the ways in which 
SSI reforms have or have not availed themselves of the avenues of influence discussed earlier in 
this paper. For the most part, policymakers and other SSI participants have invested their primary 
energies in three of the avenues, although in each case the reverberations of reform activity at the 
core of each SSI have not necessarily been felt by large numbers of classroom teachers. 

First, the SSIs have spent a great deal of time building coalitions of actors, especially at the state 
level. Prompted by NSF, these coalitions have included scientists, mathematicians, business 
people, and other community leaders, in addition to educators occupying various positions within 
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the state's system. It is unclear at present how much these coalitions, or the process of building 
them, have stimulated counterparts or a parallel process at the local level, that is, among the 
teachers, leaders, and community members whose views most directly shape what takes place in 
classrooms. Nor is it clear how sustainable these coalitions are over time. But the fact that they 
have been built at all is a significant milestone in the reform process, given that such coalitions 
combine parties that generally have not worked together before and who are not necessarily 
familiar with the collaborative style of interaction espoused by the SSI reform process. 

Second, the SSIs have promoted a set of ideas about good teaching of mathematics or science in 
a highly visible and public way. Whether or not they have successfully persuaded people to 
follow them, they have helped to make an instructional philosophy, a set of content goals, and a 
conception of the learner (e.g., as inquirer or constructor of knowledge) well known among 
public and professional audiences, including large numbers of teachers in some instances. In 
most SSI states, these guiding ideas about mathematics or science teaching appear to have been 
widely articulated for the first time as part of the public expectations for schooling (to be sure, 
these ideas have been current in professional circles for some years). In some states, the big ideas 
guiding the SSI have generated some public resistance (e.g., from religious groups worried that 
emphasis on problem solving might be a hidden form of values-oriented outcomes-based 
education). It remains to be seen how far these ideas will diffuse or remain in favor. 

Third, much time has been spent building supportive structures of one kind or another, many 
focused on ways to provide teachers with more pervasive and longer term professional support 
than can be afforded directly by the immediate grant funds. One state has made major 
investments in teacher networks; another has systematically sought to cultivate a statewide cadre 
of lead teachers; still others are creating resource centers, distance learning arrangements, or 
other mechanisms that will put expertise in content, curriculum, assessment, and instruction 
within easy reach of teachers across the state on an ongoing basis (CPRE, 1995). At present, 
these structures are in varying stages of development, and their connections to large numbers of 
teachers (or even individual teachers) appear to vary as well. 

Other avenues of policy influence are less well traveled. SSIs appear to have brought about less 
change in state requirements and regulations than might have been the case had these states not 
engaged in any attempts to align their policies prior to the start of the SSI. Similarly, although 
some attempts have been made to influence teacher preparation programs, those programs have 
not been a main priority of most SSIs, which had tended instead to put far more resources into 
continuing education for teachers currently in service; as of the end of the first year, for example, 
5% of total SSI funds had been invested in preservice education improvements as compared with 
28% in inservice activities (Shields et al., 1994). 



NSF's Systemic Reforms at the District Level 

The picture emerging of NSF-supported initiatives at the state level reflects, in part, the lenses of 
the observers, the early stage in a process involving a cast of thousands, and the primary level 
and unit of analysis (the whole state program). The result is that these reports do not yet 
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illuminate in detail how systemic reforms proceed within the boundaries of a district, no less an 
individual school or classroom. Although districts are more constrained "systems'’ within the 
larger framework of the state, districts contain a local counterpart to nearly all major elements of 
the state system, with the exception of teacher preparation, licensure, and accreditation. And 
inescapably, districts mediate between state reforms and teachers’ practice (Spillane, 1994), as 
district staff use state reforms as occasions to exercise their own agendas regarding instructional 
improvement. 

As yet there are few detailed accounts of how districts are responding to NSF-supported reforms, 
but a systematic attempt has been made to pull together lessons learned from seven school 
districts using NSF funds to overhaul elementary science education in ways that may be 
considered "systemic" (St. John et al., 1994*). Although these districts were not formally 
engaged in NSF's Urban Systemic Initiative, the systemwide scale and scope of these efforts and 
their longevity make them a useful source of insights into the dynamics of systemic approaches 
to reform in large urban districts. The account of systemic reform efforts in these districts mirrors 
some of the state-level dynamics while highlighting other processes that are more local in nature. 
For example, the importance and difficulty of vision building is underscored, as one participant 
put it, regarding the need to "reinvent" the vision: 

So we found that the time it takes to keep reinventing the vision and sharing it 
over and over ... is monumental. Trying to get people to see what you see-it is 
like waiting for whales to surface-you keep waiting and waiting and hoping that 
when it surfaces you will see all of it. But in the meantime you have to share the 
vision bit by bit with everyone who comes along. ... It is frustrating-you have to 
start all over again each time new people come aboard . . . but the time it takes for 
them to come together, and to create the vision for the first time for themselves, is 
so very important, (p. III-6) 

Like the SSIs, the districts invested heavily in professional development, especially through 
strategies that featured teacher leaders. These strategies posed several important contingencies 
that were resolved differently in the different districts: identifying and selecting the 
teacher leaders, nurturing their development as leaders (versus masterful teachers of science), 
and putting in place mechanisms and arenas in which teachers could legitimately lead their peers. 



* This report assembles not only the commentary of participants, but also the cumulative 
experience of researchers from Inverness Associates, who have worked in the seven districts 
over several years. The seven districts all received similar long-term, large-scale grants, although 
these were not formally part of the NSF’s Urban Systemic Initiative (USI). Nonetheless, these 
grants resemble those of the USI, and have involved the grantees in efforts that span many 
elements of the local system surrounding elementary science education (vision, leadership, 
professional support, curricular and logistical support, political and financial support). 

District strategies also featured curriculum development (or a process of identifying a high- 
quality curriculum) and logistical supports (e.g., to circumvent the perennial problem in 
elementary science of distributing, storing, and replenishing materials for hands-on science 
inquiry). Once again echoing state-level dynamics, district-level systemic efforts were described 
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as critically dependent on securing a diverse array of partners, some internal to the district, others 
external, who could provide political and financial support over the long term. 

Evidence Regarding Other Systemic Initiatives : The Case of the California Frameworks 

The limitations on our understanding of the connections between NSF-supported systemic 
strategies and classroom practice can be partially addressed by turning to studies of other 
systemic initiatives undertaken by states to promote ambitious new visions of science and 
mathematics teaching through a coordinated systemic strategy. In this regard, what has taken 
place in California offers an especially instructive case. Investigators have looked closely at early 
implementation of the California initiative (Marsh & Odden, 1991; Educational Evaluation & 
Policy Analysis, 1990), have carried out longer-term analysis of this reform as a whole (O’Day, 
1995) and various aspects of it (by Cohen, Ball, Peterson, Wilson, and their associates *), and 
have conducted focused studies of particular features of the policy process such as state- 
supported teacher networks (e.g., Firestone & Pennell, 1995). Other, more intensive forms of 
research have begun to dig into the curricular politics and classroom dynamics associated with 
science reform in this state (Page, 1995). From this body of work a picture is beginning to 
emerge of the fine detail of how a systemic policy such as the one enacted in California can and 
does (or doesn't) make its way into the classroom. 

Short-term Dynamics and Effects of the California Mathematics Framework Reforms 

The findings about early implementation of the California initiatives suggest some of the 
dynamics that are as yet hard to see (from written accounts) in the emerging story of the NSF 
initiatives. What we know about early implementation of the California Framework initiatives 
comes primarily from two sources. The first (Marsh & Odden, 1991) studied 14 contrasting 
schools, and classrooms within them (eight per school), to construct cross-case patterns of local- 
level implementation and system response to both mathematics and science reforms, while the 
second {Educational Evaluation & Policy Analysis, 1990) sought to capture and interpret the 
stories of individual teachers (24 teachers in three districts) responding to the state's mathematics 
framework. Though both drew extensively on qualitative sources and confined their attention to 
districts that were apparently in the forefront of local efforts to improve instruction in the two 



* This body of research, now quite substantial, is part of the multiyear Educational Policy and 
Practice Study (EPPS) conducted by a team from Michigan State University and the University 
of Michigan. This research has followed systemic reforms in mathematics and literacy in three 
states (California, South Carolina, and Michigan). Reports from that research program are too 
numerous to list in toto here; the following have been particularly helpful in providing a data- 
based picture of California's reform efforts in mathematics: Peterson, 1990; Ball, 1990; Wilson, 
1990; Cohen, 1990; Wiemers, 1990; Cohen & Ball, 1990; Darling-Hammond, 1990; Sykes, 
1990; Peterson, Prawat, & Grant, 1994; Grant, Peterson, & Shojgreen-Downer, 1996; Wilson, 
Peterson, Ball, & Cohen, 1996. These pieces are joined by numerous other conference 
presentations, chapters, or articles reporting findings from other states, and by interpretive and 
conceptual writing of various kinds. Where relevant, these other pieces have also been 
referenced in the discussion that follows. 
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subject areas, they approached the problem with somewhat different designs and units of 
analysis, as well as different premises about what to look at. 

The two studies came, at a first read, to apparently opposite conclusions about the initiative's 
impact, with the first study proclaiming substantial successes for the reform (i.e., rapid adoption 
of reforms; changes in district, school, and classroom practices realized in relatively short time) 
while the second found more failure than success (i.e., teachers making partial, superficial, or 
negligible change in their practices, while at the same time often believing they were changing in 
more fundamental ways). However, read another way, the findings of these two studies offer a 
more complementary composite account, as I have noted at greater length elsewhere (Knapp, 
1995): 



Taken together, [the two studies] may tell the following story: the curricular 
reform policy was successful during its early years in activating a large number of 
actors in regional networks, districts, and schools on behalf of an ambitious 
mathematics [and science] education reform. Various factors, both those related to 
the policy and those embedded in local contexts, converged in “ripe” districts 
(those with sufficient capacity and will to undertake this reform) to encourage 
large numbers of teachers to pay attention to the policy and attempt to realize it in 
the classroom. There, the policy has met with more mixed success-so far. 

Teachers’ efforts to put the policy into practice have succeeded in changing the 
more easily altered aspects of practice but have been limited by the teachers’ 
grasp of and beliefs about mathematical knowledge itself, by their capacity to 
visualize how a new conception of knowledge translates into teaching, and by the 
momentum of teaching traditions. These same limitations apply in varying 
degrees to the people who are in a position to support teachers’ efforts, including 
mentor teachers, school and district administrators, curriculum coordinators, and 
others-the kinds of people activated by the reform policy, (p. 18) 

In particular, the two studies highlight different avenues of influence by which the reforms might 
be reaching classrooms. The Marsh and Odden study focused on the local policy system as the 
primary unit of study; furthermore, it conceived of the "policy" as a complex, evolving course of 
action, lodged within a set of related policy initiatives, with a history prior to its enactment. The 
rapid "success" of the policy in its adoption by the districts and schools under study was 
attributed to (1) capacity built at the school and district level prior to the enactment of the state 
policy; (2) the formation of new coalitions at the district level, favorable to the changes in 
mathematics and science, which bridged the professional interests and expertise of teachers and 
the bureaucratic interests and know-how of district officials; (3) the existence of supportive state 
initiatives (e.g., a mentor teacher program, a school improvement grant program); and (4) 
creative use of resources at the school level to help sustain the use of the reforms. 

The predominant picture to emerge from this study is one of a great deal of activity at both 
district and school levels, activated in part by the systemic reform policy, which had hastened 
teachers' awareness of the reform visions, supported teachers' attempts to realize the reform 
vision, and bought about widespread "use" of most curricular themes found in the two state 
curricular frameworks. By aggressively promoting the spread and visibility of big ideas and by 
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mobilizing coalitions of actors in relation to these ideas, the policy apparently took root in the 
already fertile soil of the districts under study. 

But this research is inarticulate about what "classroom use" might mean and precisely what ideas 
might have made their ways into teachers' heads, their instructional decisionmaking, or the actual 
enactment of curriculum in daily lessons. The more intensive observation and interviewing of 
individual practitioners in the second study helps to see what was and wasn't taking place in 
certain classrooms and how these practices might be linked to the state's reform efforts. 

The five case accounts of individual teachers' responses to the mathematics framework (Ball, 
1990; Cohen, 1990; Peterson, 1990; Wiemers, 1990; Wilson, 1990) tell a more complex story 
concerning intended effects, unanticipated consequences, and noneffects of policy. Across all 
five teachers, there were distinct differences in how well they grasped what it was all about and 
how much the policy had touched their practice. In one case, the teacher (Mrs. O) made what 
appeared to her to be revolutionary changes in her practice, which nonetheless missed many of 
the deeper, more subtle intentions of the reform vision embodied in the state framework (Cohen, 
1990). In another, the teacher (Carol Turner) appeared relatively untouched by the reform vision, 
in some ways already conforming to its expectations and in others subtly contradicting it (Ball, 
1990). In a third, the teacher (Cathy Swift) had grafted new practices (like group work and the 
use of manipulatives) onto a traditional skills-oriented approach with which she felt comfortable, 
yet had begun to entertain questions about her practice as she noticed the discontinuities between 
the new and old elements in her teaching (Peterson, 1990). 

In all but one case (who flatly rejected most of what the reform was promoting), the teachers 
showed signs of taking on some of the ideas, trappings, and procedures of the reform vision 
while retaining much of what was familiar to them. Their capacity to change their ways of 
teaching mathematics as profoundly as the Framework vision urged were clearly shaped by their 
limited grasp of, and prior experience with, the mathematics the Framework expected them to 
teach (Cohen & Ball, 1990). While the study results are not too explicit on this score, the 
teachers' way of implementing reform intentions appeared to be further limited by the kind of 
support they received, the array of resources at their disposal, and the apparently modest 
understanding of the mathematics reforms by those (e.g., administrators, curriculum specialists) 
with whom they would most likely be able to interact. 

If they do little else, the case accounts demonstrate various ways in which supposedly aligned 
systemic elements (e.g., curricular Framework expectations, textbooks) might not have the fully 
reinforcing effect at the local level that state policymakers had assumed. 

There is no way of knowing how well the findings of the two studies mesh. Most likely, the 
teachers in the Marsh and Odden study, who reported making extensive use of the new 
curriculum themes in their classrooms, were engaged in the same kind of hybridization described 
by the second study and for similar reasons. Alternatively, it is possible that the teachers in the 
first study were better grounded in mathematics and science and were working in a more 
supportive environment. In that case individual case accounts of their practices might have come 
out quite differently from the ones noted above. 
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Longer Term Dynamics and Effects of the California Mathematics Framework Initiative 

These studies of early implementation beg many questions about the longer term effects of 
California's systemic initiative, the range and depth of impacts across a wider variety of 
individuals and localities, and the precise dynamics of change in the intergovernmental space 
between reform source (at state level) and classroom. Fortunately, longitudinal findings from the 
Educational Policy and Practice Study (EPPS), along with some more recent work, begins to 
shed light on these matters. 

The individual teachers studied by the EPPS team have been followed over time, and results are 
beginning to be published. For example, a recent account (Grant, Peterson, & Shojgreen- 
Downer, 1996) describes what teachers in one California elementary school have learned across 
four years of the reform process. For one thing, it is clear that the teachers’ learning did not 
necessarily stop with the early discoveries (or lack thereof) documented by the early 
implementation research. In short, the teachers followed in this research did continue to learn, 
although not necessarily in ways intended by the reform policy, nor necessarily because of it. 
When the state Frameworks first made their appearance on the teacher’s (Cathy Swift) horizon, 
they helped to stimulate her awareness of and questions about possibilities for teaching and at the 
same time helped her recognize the contradictions between the expectations of reform vision and 
the conditions in which she found herself working (a structured basic skills-oriented curriculum, 
frequently tested, which afforded little time, in her view, for the kinds of learning experiences 
that would encourage understanding). Four years later, with a change in teaching assignment and 
repeated exposure to new ideas and curricula, Cathy had made strides in her ability to put key 
elements of the vision into practice, for example, in her use of manipulatives. By working 
alongside her students and interviewing them to ascertain how they were thinking about their 
work, she had begun a critical process of discovering how her students were constructing 
mathematical understanding (Grant et al., 1996, p. 526). Her learning, and that of the other two 
case study teachers in this school, prompted the researchers to suggest that the state’s systemic 
reform efforts could be judged a success, in some respects: 

Each teacher has moved her or his practice in the direction advocated by the 
reformers: new content is taught, new instructional representations are used, new 
forms of classroom organization are in place, all students are participating. As 
frameworks, textbooks, tests, and inservices have changed, so too have the 
teachers’ practices. Advocates of systemic reform could point to these cases as 
evidence that the theory works, (pp. 531-532) 

But the longer-term success of systemic mathematics reform in this school is not without 
qualifications. The three teachers in the school made different sense of the reform expectations- 
for example, regarding what precisely was to be understood by students (e.g., abstract algorithms 
or concrete representations of mathematical ideas). These differences among the teachers make 
the policy’s effects less coherent than in its conception, at least at this stage of their response to 
the policy (it is conceivable they are on their way, by different routes, to a more mutually 
consistent vision of teaching). Furthermore, all teachers’ learning was still limited by their lack 
of knowledge of the mathematics they were teaching, despite many opportunities to learn it more 
deeply and some progress in this direction. And all teachers encountered conditions in their 
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working environment, such as testing programs that did not align with the frameworks or 
persistent program philosophies based on wholly different principles, that contradicted the 
directions advocated by the reform policy. 

The content and extent of teachers’ learning and their capacity to introduce it into their work in 
the classroom were heavily dependent on the immediate circumstances of their professional 
assignment, collegial peer group, and nature of opportunities for learning. Here, the case 
accounts point to both numerous chances to come to know what the state’s reform visions 
advocated, see it in action (e.g., in demonstration lessons), and practice it with access to advice 
and support. But at the same time, these opportunities were themselves constrained by the vision 
and skills of those who demonstrated, advised, or taught the teachers, as the researchers 
commented: 

The content of [the teachers’] professional development opportunities focused on 
mathematics for all students but failed to delve deeply into relevant issues of 
culture, language, and ethnicity. And despite all the talk about new conceptions of 
mathematics teaching and learning for all students, the pedagogy of teaching 
teachers reflected traditional didactic practices. In virtually every instance, Cathy, 

Carlos, and Crystal experienced teaching based on “modeling,” with an emphasis 
on initiating and practicing the behavior by “walking through it.” Little attention 
was given to teachers’ thinking .... The afterschool mathematics leaders, for 
example, scripted mathematics lessons for the teachers and gave them games, 
activities, and materials to be used as is with their students. Moreover, teachers’ 
different understandings did not surface and so were not probed in these 
situations. (Grant et al., 1996, p. 535) 

As this quote implies, the depth and range of policy impacts on teachers were crucially 
dependent on the way the policy influenced — and was influenced by — the individuals and 
processes that lie between policy and classroom practice. Other EPPS studies shed light on how 
district-level administrators and other actors (e.g., at the state level) mediate and interpret the 
original policy intentions. There is evidence the state’s reform initiative engaged some district 
administrators in a process of discovery and advocacy, as in accounts of central office 
administrators in the district in which Cathy, Carlos, and Crystal taught (Peterson, Prawat, & 
Grant, 1994). One of four administrators tracked in this study, Christine Lytle, a person in 
charge of staff development, exemplifies the considerable shift in approach and underlying 
premises that could come in the wake of a major reform shift. Where once she approached 
teaching and the pedagogy in very traditional terms, emphasizing basic skills instruction and 
uniform “effective teaching models,” four years into the reform: 

Lytle sees that for teachers to learn and change, she needs to offer a different 
form of pedagogy — one that allows for more individual exploration and problem 
solving. Moreover, Lytle no longer believes that “one model fits all” for either 
teachers or children. This change in Lytle’s thinking/practice as a teacher and 
district administrator makes things much more uncertain and problematic for her. 

There are no longer any easy answers. But Lytle seems to understand that this is 
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an important step in getting teachers to understand and enact deep changes in their 
practice, (p. 32) 

As the researchers construed it, the changes in Christine Lytle and others in this district were 
possible because the central office staff approached the challenges posed by the curricular reform 
as opportunities to learn and, in taking advantage of these opportunities, they “strove to make 
sense of and enact the pedagogical reforms in their own policies and practices” (p. 33). Across 
all the sites studied (in and out of the state), the EPPS research team encountered a range of 
responses among such individuals. Where policymakers or other intermediaries became or 
remained open to new learning implied by the reform vision, then the reform was more likely to 
make progress. The following summarizes observations of state-level officials in charge of 
California’s mathematics reform: 

In our early interviews it wasn’t clear — to any of us — that the process of 
transforming mathematics teaching in California would also entail policymakers’ 
transformation. Yet that is exactly what we have witnessed; for over time, we 
have seen significant changes: in their language, organizing principles, 
assumptions, knowledge, and beliefs (Wilson, Peterson, Ball, & Cohen, 1996, p. 

4) 

The process of learning and adjustment of policy implementation strategies has been ongoing in 
California, and it reflects an ever expanding search for mechanisms, supportive structures, and 
other means that can enrich the environment surrounding the mathematical teaching practice in 
the state. That search has led to some substantial shifts and additions to the original policy 
strategy: for example, a shift away from the original strategy of relying on textbooks toward 
“replacement units” (specially developed units that captured more fully the vision of the 
curriculum Frameworks than did commercially published materials, combined with intensive 
staff development) until such time as more appropriate textbooks appeared (O’Day, 1995). 

The Broader Course of Systemic Reforms in California 

Other studies shed further light on several aspects of California’s systemic reforms-research on 
the systemic strategy as a whole (O’Day, 1995, as part of Goertz, Floden, & O’Day, 1995) on 
state-sponsored teacher networks (Firestone & Pennell, 1995) and on implementation of the 
Science Frameworks in “ordinary” high schools (Page, 1995). 

These studies thicken the picture of systemic reform response dynamics in several ways. First, in 
selected “high-capacity” districts and schools-which probably resembled or even exceeded the 
capacity of those examined in early and subsequent research-the Frameworks in 1993-94 appear 
to represent “a visible and major influence on the curriculum and instruction” (O’ Day, 1995, 
p. 25). The influence might be direct, as in one district where state-developed replacement units 
formed the core of the recently revamped project-based curriculum at the elementary level, or 
more indirect in another district where teachers participated extensively in various professional 
development activities related to the Frameworks. The researchers noted in this case that, while 
at the middle school level, not all teachers were enthusiastic about the lack of emphasis on skills, 
“all the teachers seemed to see the Framework as a ‘good foundation’ and placed considerable 



23 



ERIC 



31 



emphasis in their instruction on problem-solving and mathematics application as outlined in the 
Framework” (O’Day, 1995, p. 26). Furthermore, all teachers who had direct contact with the new 
performance-based assessment system, which was aligned with the frameworks, saw it as a step 
in the right direction. 

To account for the pattern of implementation, a series of factors were identified that distinguish 
high-capacity settings such as these from others and show how such settings establish the local 
level groundwork for the positive response of the teachers to reforms (O’Day, 1995). All four 
schools examined in this research were actively engaged in restructuring and had gotten 
themselves past the preliminary steps in such processes. Teachers in the schools valued 
collaboration and had formed within-school communities of learning that were an important 
point of reference for them, perhaps more than out-of-school links with professional networks. 
Teachers “emphasized the importance of having a critical mass of teachers and a school culture 
committed to and supportive of change” (p. 32). The schools and the districts in which they were 
housed offered numerous opportunities for professional development that was engaging, 
intellectually challenging, ongoing, and respectful of the teachers as professionals. In addition, 
there were numerous other avenues for learning-such as teachers participating in scoring student 
assessments, deliberating about textbook adoptions, or using replacement units. 

While in-school collegial relationships have an important role to play in reinforcing (or 
inhibiting) reform intentions, external professional linkages may also play a central role. State- 
sponsored teacher networks were added to the original course of action promoting the 
Frameworks’ vision through the creation of the California Subject Matter Projects, which offered 
access to intensive summer institutes and related follow-up in eight subject areas, mathematics 
and science included. Case study research on selected institutes and a sample of 30 teacher 
participants in the summer of 1994 and the 1994-95 school year (Firestone & Pennell, 1995) 
suggests the effects on teachers who participated were substantial-including varying degrees of 
deeper learning about subject matter and pedagogy, psychic support and rejuvenation, 
opportunities for leadership development, and a kind of empowerment, in the sense that 
networks became informal lobbying organizations through which teachers communicated their 
views about particular legislative initiatives. At the same time, the relatively small numbers of 
teachers statewide who participated (estimated at less than 2% of all teachers in the state) and the 
loose connection of network activities to policy aims limit the aggregate contribution of 
networks to the overall policy goal. 

Though they help to visualize both the nature of teachers’ responses to systemic reforms and 
some of the ways in which systemic reforms might make their way through the system, the 
published work from the EPPS and CPRE programs of research leave unanswered a number of 
questions about how the different types of teacher responses might be distributed among teachers 
across the state;* how differently teachers and the system might be responding in science versus 
mathematics; how high schools might be responding to systemic reforms in either subject area; 
and what the response to systemic reforms might look like in schools other than those with high 
or moderately high capacity. 

Preliminary results from ethnographic research on the implementation of systemic science 
reforms in several “ordinary” high schools in southern California begin to answer some of these 
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questions, by painting a picture of implementation dynamics in lower-capacity settings. A recent 
report of selected aspects of this work (Page, 1995) highlights the way schools and districts went 
about offering a more “integrated” form of science instruction in ninth- and tenth-grade 
classrooms, in line with the emphasis placed on integrated science in the Science Framework for 

California Public Schools (California Department of Education, 1990). The overall effect at the 
classroom level in the first year of implementation was summarized as follows: 

In classrooms, Integrated Science I [for ninth graders] developed haphazardly, as 
an incoherent amalgam of bits and pieces of the physical, chemical, and biological 
sciences; the disciplines the teachers were accustomed to teaching received the 
most emphasis. “Hands-on,” “practical,” “lab-based” science was trivial; teacher 
talk, textbooks, and worksheets prevailed, accompanied by student high jinx. 

Curriculum was neither “innovative,” “interdisciplinary,” nor “coordinated,” but 
what one teacher aptly described as “on-the-fly” selections from resources already 
developed for traditional discipline-based courses. In short, “integrated science” 
had the content and form usually attributed to lower-track classes. (Page, 1995, p. 

25) • "• 

The context for this pattern of science teaching reveals a constellation of forces quite unlike the 
high-capacity settings studied by CPRE researchers. The district in question allocated few 
resources to support the introduction of integrated science implementation and reserved most of 
these resources for administrative level work, not classrooms. Furthermore, these efforts took 
place in the absence of a fully developed set of integrated curricular materials, as in the case of 
Montana's SSI initiative, discussed earlier. Teachers received at most three or four released days 
to develop the year-long course and a very meager budget for instructional materials. The 
schools in question did not appear to have collaborative staffs, and tensions fractured the 
implementation 

effort along departmental fault lines. District-developed guidance was not well received by 
teachers, “perhaps because they had no input or because it was too generic” (p. 25). Coercive 
measures were undertaken to force teacher compliance. At the same time, parental opposition 



* Regarding the distribution of responses across the state, EPPS has conducted a statewide 
large-sample survey of California elementary teachers and is in the process of analyzing it, both 
to determine broader estimates of response to reform and to locate the individual longitudinal 
cases within it. Conference presentations (e.g., symposium at the 1996 annual meeting of the 
American Educational Research Association entitled ’’Teacher Learning from Mathematics 
Reform: What Can Survey Research Tell Us?") have begun to make public patterns of response- 
for example, that substantial proportions of the state’s teachers are familiar with the state's 
Framework and feel positively toward it, yet at the same time have not abandoned more 
traditional teaching practices. 

developed (with some assistance from skeptical teachers), and the district was forced to make the 
new integrated science course an option rather than the basic curriculum for all students in high 
school. 



Evidence Concerning Systemic Reforms in Other States 



25 




33 



While the case of California is highly instructive, its lessons regarding systemic reform must be 
understood in the context of other states, which differ from it in many respects. In some 
respects, the large size of this state, its complexity, the rapidity of demographic changes in the 
student population, and the less than favorable political and fiscal trends may make it a relatively 
hostile environment for systemic reforms to take root, even though the state's reform efforts have 
been in place for nearly a decade with consistent strong leadership from the top across most of 
those years. Fortunately, a fair amount of evidence regarding systematic reform efforts (other 
than NSF-supported SSI initiatives) is accumulating from states as disparate as Michigan, South 
Carolina, Vermont, and Kentucky to round out the picture of policy-practice connections 
suggested by California's experience. Most of what we know about systemic reforms aimed at 
mathematics and science education in these states comes once again from studies undertaken as 
part of the EPPS and CPRE research programs (e.g., Price, Ball, & Luks, 1994; Grant, 1995), 
supplemented by other investigations (e.g., Spillane et al., 1995). 

In the main, research from these studies confirms the broad contours of reform impact described 
above for California. Early responses to the reforms at the classroom level display similar 
patterns of partial implementation of reform, while preserving much of the traditional basis for 
practice that reformers seek to alter. Furthermore, these patterns appear to be in place for many 
of the same reasons, as the following analysis of a fifth-grade teacher in Michigan suggests: 

The textbook is indisputably central to Mr. Burch's [mathematics] teaching. 

Therefore, Mr. Burch's practice may at times reflect elements of the reforms — to 
the extent that the new text series itself embodies them. But the textbook is not the 
sole determinant of what Burch does in his classroom. In following the text, his 
teaching is significantly shaped by conventional habits, orientations, and beliefs. 

His own knowledge of mathematics affects the way he reads and uses the book. In 
this lesson, Burch had the reins and controlled the talk; what was talked about 
centered on small facts and right answers to be memorized. Neither teacher nor 
students discussed the ideas or raised the questions on their own; instead, they 
followed the book's script together. And students' turns were small and 
constrained. Burch's views of learning, his notions about mathematics and 
measurement, his perceptions of his students and what they need — these came 
together in Burch's practice. (Price et al., 1994, p. 14). 

This teacher worked in a district in which higher level administrators placed relatively less 
priority on mathematics reform than on other subject areas and had correspondingly less 
understanding of the new mathematics called for by the state's reform initiative. Other districts 
might muster more resources supporting mathematics reform than in this instance (although there 
are indications across states that mathematics reform often gets short shrift by comparison with 
reform in literacy, as suggested by Ball & Cohen, 1995). As in California, there are wide 
variations in the way schools and districts have approached mathematics or science reform ideas, 
with predictable consequences for the pressure to change and resources for change experienced 
by teachers. 
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But while classroom practices have not always been touched, teachers' learning has, in ways that 
mirror the variety in responses documented in California. State reform strategies appear to have 
activated many district-level actors, who have championed their own versions of reform ideas 
among the educators within their purview. And they have also stimulated teachers to question 
basic assumptions underlying their practice and to try, even if by partial steps, new ways of 
guiding learning in these subject areas (e.g., Grant, 1995). 

But the differences across states help bring certain patterns into sharper relief. First, the 
formulation of the state's strategy itself, in combination with varying traditions of local control, 
result in different configurations of local instructional guidance (see, for example, Goertz et al., 
1995, regarding the contrast among California, Michigan, and Vermont). Some states leave more 
room than others for local initiative in developing instructional guidance; some project relatively 
coherent messages, while others send decidedly mixed messages, as in instances where 
curriculum mandates are not accompanied by fully supportive assessment policies (David & 
Goren, 1993). Second, the variation across states (and within them, across and within districts) 
helps to dramatize the fact that attempts to bring about coherent policy may be unwittingly 
increasing the diversity of approaches to mathematics and science education-in a sense, 
decreasing the coherence of the system-at least in the short term (Cohen, 1995). Third, the fact 
that common themes emerge across disparate state contexts signals the enduring importance at 
the teachers' level of particular kinds of support for reform, as in the role that "communities of 
practice" outside the school appear to play in sustaining teachers' investment in reforms (Goertz 
et al., 1995). 



Interpreting the Dynamics of Systemic Reform: Three Vantage Points 

While the evidence is not as complete as we might like, it offers some provocative insights into 
the forces, conditions, and processes that intervene between reform policy and the classroom. At 
the same time, the evidence begs for interpretation. Three conceptual vantage points, each 
deriving from a different literature base, offer distinct although related understandings of what is 
going on in this instance. The first, the oldest viewpoint, approaches systemic reform as a 
problem of innovation and change and concentrates on how new practices diffuse and are 
replicated in a large number of settings. The second viewpoint focuses on the dynamics of 
reform policy implementation and draws attention to the way policy intentions interact with 
contextual forces throughout a multilevel policy system. The third considers the learning that 
individual professionals throughout the system must do, and what organizations or the system as 
a whole must do, to realize the intentions of ambitious reform agendas in these subject areas. I 
discuss below how each vantage point might interpret what appears to be happening in recent 
instances of systemic reform, and what these interpretations would suggest will continue to 
happen as time goes on. 

Systemic Reform as Innovation and Change 

A long-established way of viewing educational reforms focuses attention on innovation and 
change dynamics. In this view, systemic reform in mathematics and science education represents 
both a discrete innovation introduced by the leadership of educational systems (e.g., states, 
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districts) and a sophisticated planned change strategy designed to encourage adoption of the 
innovation by teachers in schools. 

Innovation theory and research have been around for decades, growing out of an extensive 
tradition of such studies in the private sector and in the field of communications on the diffusion 
of innovations (Rogers, 1962). Analysis of innovation processes took root in education during 
the early 1960s as many sought to revitalize education (see Miles, 1964, for a summary of this 
work). From that point forward, this line of thinking was advanced by case study research on the 
various innovations of the day (e.g., Gross, Giacquinta, & Bernstein, 1971, regarding open 
classrooms in elementary schools), the stages and processes of innovation adoption (e.g., Hall & 
Loucks, 1977, 1978, and subsequent related studies), and the impact of change agents in 
increasingly large-scale interventions (e.g., Berman & McLaughlin, 1978; Crandall et al., 1982). 
Synthesis and formulations of the key ideas emerging from these lines of work have appeared 
periodically (e.g., Fullan & Pomfret, 1977; more recently, Fullan & Stiegelbauer, 1991; and 
Odden, 1995). 

In its classical form, this perspective concentrates on the nature of educational innovations, the 
processes by which resistance to them can be overcome in an initially reluctant organizational 
system, and the dynamics of innovation adoption. The tradition presumes a relatively discrete 
innovation — typically, a practice, product, process, or organizational arrangement, originating 
from sources exogenous to the adopting system — that is to be “diffused,” “disseminated,” or 
“introduced” to “users” throughout the system. “Change agents” (variously conceptualized as 
“change facilitators,” “interventionists,” and “linkage agents,” among others) promote the 
awareness of the new practice, product, or arrangement and encourage its use through a 
dissemination strategy that combines various incentives and supports. The innovation is assumed 
to encounter varying degrees and forms of resistance, a predictable state of affairs, given the 
assumption that systems seek to maintain equilibrium. Resistances are generated by the 
characteristics of innovation itself (e.g., how complex or costly it is), its compatibility with the 
adopting organization (e.g., how congruent it is with prior practices and users’ knowledge), and 
the sophistication of the change strategy (how well change agents anticipate the state of the 
system and its readiness for change). This perspective further identifies distinct stages in the 
process of adoption, for example, the “stages of concern” exhibited by individual users 
(identified by Hall & Loucks, 1978) or the overall movement of an innovating system from an 
“initiation” stage to “initial implementation” and subsequently to “continuation” (as summarized 
by Fullan & Stiegelbauer, 1991). 

What Innovation Perspectives Explain 

At first glance, the perspective offers a reasonable account of much that is taking place in the 
systemic reform of mathematics and science education, based on the evidence available so far. 
The innovation itself (intellectually challenging mathematics and science teaching) is complex 
and entails costs in time and resources that often exceed what is currently available at the 
operating level. The adopting systems (large districts, whole states) are large and complex, as are 
the change strategies (systemic alignment, etc.) themselves. Given these facts, it is logical that 
large amounts of energy would have to be expended in awareness building among various 
constituencies. 
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This perspective anticipates significant resistances under such circumstances, or at least 
widespread concerns among users over what the innovations might mean for their personal 
investment of time. In adopting organizations with the most hospitable climates, requisite 
resources, and teachers whose practices most closely resemble what systemic reforms seek, 
resistances are likely to be lower. Given some sustained pressure for change and at least some 
environmental support, this perspective would assume that levels of concern will gradually 
lessen and increasing numbers of teachers within each relevant unit (e.g., school) will adopt the 
practices, as appears to be happening, at least in high-capacity schools. Over significant periods 
of time (variously set for complex reforms between 5 and 100 years), as a “critical mass” of 
users develops in each adopting unit, the new teaching practices are likely to become well 
ensconced. 

What Innovation Perspectives Do Not Explain 

But there are other aspects of the systemic reform story that an innovation perspective, so 
conceived, does not explain so well. First, the perspective assumes a relatively discrete, 
describable innovation, a product, practice, process, or arrangement that the originators 
understand fully, so that the task is one of transfer of ideas rather than the ideas themselves. In 
this sense, the perspective treats teachers as “users” of ideas picked up from other people. But 
teaching mathematics or science for understanding is far more emergent and less specified than 
the innovation construct generally implies; and at the same time, it places more emphasis on 
teachers developing an internalized process of constructing ideas about their practice. While 
there are some guiding principles and articulations of them, the “innovation” in this case refers to 
a much looser combination of ideas about learning based in constructivist thinking, premises 
about the purposes of schooling, assumptions about preferable practices, and theories of how 
instructional practices interact with students’ learning. 

Second, the innovation perspective does less well with multiple innovations than with single 
ones (Fullan & Stiegelbauer, 1991), yet the local settings responding to systemic science and 
mathematics reforms were often engaged in multiple reforms (e.g., Grant et al., 1996). As a 
consequence, recent thinking in this line of work leads some to urge reformers to concentrate 
attention on the institution 'as a whole (Fullan & Stiegelbauer, 1991) or the school as a whole 
(e.g., Odden, 1995). 

Third, the perspective focuses most of its attention on the “end-users” (teachers) rather than the 
intermediaries and the way they construe the innovation, adapt it, or redefine it. Some attention, 
however, has been paid to the developmental progression that individuals in a position to guide 
or support teachers might experience in their own concerns about and use of innovations-for 
example, school principals (Hall, Rutherford, Hord, & Huling- Austin, 1984) and more 
generically “change facilitators,” that is, any persons who aid in the implementation of the 
innovation (Hall & Hord, 1987). 
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Systemic Reform as Policy Implementation 



A second perspective, built around the central construct of “policy implementation,” sheds light 
on the systemic reform story in a somewhat different although complementary way. Rooted in 
political science research, this line of investigation has developed over the past several decades 
from seminal examinations of public policies initiated in the 1960s (e.g., Pressman & Wildavsky, 
1973; Bardach, 1977). Applications to educational policies date from the mid-1970s, set in 
motion in large measure by the Rand Change Agent Study (Berman & McLaughlin, 1978) and 
continued through various studies of educational policies, especially those emanating, first, from 
the federal level (e.g., Murphy, 1971; Barro, 1978; Hargrove, 1981) and then from the state level 
(e.g., Moore et al., 1991; various CPRE-sponsored studies of state reform implementation in the 
mid- and late 1980s). The lessons learned from these lines of research have periodically been 
summarized (e.g., Peterson, Rabe, & Wong, 1986; McLaughlin, 1991; Fuhrman, Clune, & 
Elmore, 1988; Odden, 1991; Elmore & Sykes, 1992). 

Like innovation research, policy implementation studies are concerned with change, but their 
focus is typically on the larger dynamics by which policies make their way through an 
intergovernmental system and are reinterpreted in the process. The starting point for such studies 
is thus an enacted policy, a publicly announced set of intentions, often combined with an 
allocation of resources and other provisions designed to motivate and sustain change-oriented 
activities lower down in the system. This line of thinking pays special attention to the interaction 
of policies with their contexts at each stage in the process, for example, at the state level (Moore 
et al., 1991) and at the local level (e.g., Weatherly & Lipsky, 1977; Knapp et al., 1991) and 
argues, in effect, that contextual factors exert ultimate control over the actual direction of policy 
as “delivered” to “service recipients.” Some scholars working in this tradition, even talk about 
the professionals at the lowest level of the system as the “policymakers” (Lipsky, 1980) or 
“policy brokers” (Schwille et al., 1984). The upshot for policy is that, much as it may influence 
the settings toward which it is directed, it also is changed in the process, leading to what was first 
described as the “mutual adaptation” of policy and setting to each other (Berman & McLaughlin, 
1978). 

By paying close attention to the role of contexts in shaping policy implementation, this tradition 
is more concerned with explaining the variation in local response to policy than the central 
tendencies of the policy. In particular, policy implementation research offers a beginning 
vocabulary for talking about the kinds of forces that make the most difference in the 
implementation story — the “capacity” of implementing agencies as well as their “will” to put the 
policy into practice (McLaughlin, 1991). Lines of research branching off of the policy 
implementation tradition have taken the matter farther by, for example, exhaustively studying the 
nested contexts in which teachers work, only some of which can be touched by policy actions 
(McLaughlin & Talbert, 1993). Other work has elaborated the ways in which policy attributes — 
in particular, the power and authority associated with policies, but also their consistency and 
prescriptiveness — might make a difference in the ways teachers attend or respond to instructional 
policies (Floden et al., 1988). 
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What Policy Implementation Perspectives Explain 

This line of investigation helps to draw attention to the dynamic relationship between a systemic 
reform policy and the immediate context of its enactment, such as a state government, thereby 
pointing out the many ways that a systemic reform strategy might evolve, subject to the political 
culture, fiscal environment, prior policy commitments, and other features of the state policy 
environment. Thus, the rapid progress of the systemic reform initiative in Montana can be 
partially understood in terms of the state level forces and conditions that coalesced around a 
particular, somewhat limited, conception of reform, one focused on high school mathematics 
teaching. Forward motion was not impeded by substantial tugging and hauling between the main 
state-level actors. 

The perspective also helps to describe more and less hospitable local environments— those in 
which the capacity and will of local actors is such that systemic ideas are likely to take root, and 
other settings in which this outcome is less likely (the perspective is not necessarily clear about 
how settings come to develop high or low capacity, however). The “high-capacity” schools and 
districts in the CPRE research are good examples of settings in which contextual forces of 
various kinds maximize the receptivity to ideas about mathematics or science instruction that 
maximizes intellectual challenge and at the same time increases the ambiguities involved in 
teaching. Similarly, the apparent capacities (and will) of district-level actors in Cathy Swift’s 
district help to push the professional development environment in the direction called for by the 
mathematics and literacy reforms (Peterson et al., 1994), while their counterparts in an 
“ordinary” district do not create such an environment in support of integrated science (Page, 
1995). 

What Policy Implementation Perspectives Do Not Explain 

Helpful as it is in understanding the varied nature of local implementation and — in broad 
terms — the way local contexts limit the reach of systemic reform policies, this perspective does 
not take us deep into the meaning of such constructs as “capacity,” “will,” or “mutual 
adaptation.” Where does capacity come from, for instance? How does high capacity develop in 
places such as those studied by the CPRE team and how does or can the policy contribute to that 
development process? Are capacity and will reciprocally linked, and if so how? While policy 
researchers have long recognized capacity development as one basic strategy underlying policy 
(McDonnell & Elmore, 1991), scholarship focused on the implementation of policy has less to 
say about this fundamental feature of the context surrounding the implementation process. Too 
easily, contextual conditions in the variety of case examples studied so far are treated as static 
givens, rather than as dynamic, developmental features of an environment that is itself evolving. 

In the same vein, policy implementation research runs the risk of ignoring the evolution of policy 
ideas themselves, that is, all that goes on in the initiation and formulation of policy prior to 
enactment or during its subsequent reformulation. Such an assumption belies the many changes 
that may take place in the “course of action” that constitutes a policy (as defined for this paper) 
and may presume greater consistency over time than is likely to pertain in actual cases. 
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Systemic Reform as Professional and Organizational Learning 

A third perspective, focused on professional and organizational learning, explores some of the 
features of the systemic reform story about which the previous two perspectives have less to say. 
This perspective weaves together three streams of research; the first, a branch of policy 
implementation research that seeks to understand compliance behavior (e.g., Berman, 1982); the 
second, an ongoing line of investigation in organization research that seeks to conceptualize and 
describe “organizational learning” (Cohen & Sproull, 1995); and the third, a line of thinking 
closely identified with the EPPS team across the past decade that concentrates on the 
professional learning of policy actors and the “pedagogy” of policy (e.g., Cohen & Barnes, 1993; 
Ball & Cohen, 1995). Taken together, these bodies of work offer ideas on both the learning that 
individuals must engage in to realize the aspirations of recent systemic mathematics and science 
reforms and the kind of “learning” that takes place at the organizational and institutional level in 
schools, districts, and larger systems as the reform ideas filter into practice. 

At the heart of this perspective are two sets of ideas. The first set concerns individuals 
confronted with a reform vision that presumes knowledge, skills, and even attitudes they may not 
possess and are unlikely to have experienced in their own learning as a student or work as a 
teacher. Thus, this perspective argues that, if systemic reforms in mathematics and science are to 
be realized fully in classrooms, teachers must engage in a long-term process of learning that 
resembles the kind of constructivist processes that they and reformers hope will characterize 
students’ classroom experiences. Individuals at other levels of the system who are in a position 
to support teachers’ work directly or indirectly — principals, curriculum coordinators, 
professional development organizers, superintendents, state agency officials, and even Board 
members and parents — face a similar learning challenge. They, too, must grasp what 
mathematical ideas might underlie subtraction and long division or what integrated science might 
imply about disciplinary boundaries they have long held sacred. 

By invoking learning as a central construct, this perspective also focuses on “teaching,” 

“learning opportunities,” and “resources for learning” (Ball & Cohen, 1995). In other words, 
what do policies and policymakers do to instruct actors throughout the system regarding the 
meaning of the policy and how to put it into practice? What opportunities are constructed for 
learning to take place, and how are these supported? In principle, the structure of the policy 
itself, the manner in which it is conveyed to those who are to carry it out, and its provisions for 
ongoing support and guidance over time are all subsumed within its “curriculum” and 
“pedagogy.” 

A second set of ideas concern “learning” at the organizational or institutional level. While it is 
easy and tempting to treat this as the sum total of learning by individual professionals, scholars 
working within several traditions have pointed to a whole that is greater than the sum of the 
parts, although they have done so differently, depending on their paradigmatic assumptions and 
precise research focus. Early attention to the problem of compliance by implementation 
researchers treated “organizational learning” as the institutional memory embedded in 
organizational routines and rituals that develop as schools or districts respond to new policy 
initiatives (Berman, 1983). Work related to this set of ideas has continued to explore the concept 
theoretically and empirically in a variety of organizational settings, mostly outside education and 
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the public sector (Cohen & Sproull, 1995). One mainstream definition of the central construct is 
as follows: 

Organizations are seen as learning by encoding inferences from history into 
routines that guide behavior. The generic term "routines” includes the forms, 
rules, procedures, conventions, strategies, and technologies around which 
organizations are constructed and through which they operate. It also includes the 
structure of beliefs, frameworks, paradigms, codes, cultures, and knowledge that 
buttress, elaborate, and contradict the formal routines. Routines are independent 
of the individual actors who execute them and are capable of surviving 
considerable turnover in individual actors. (Leavitt & March, 1996, p. 517) . 

The key idea in this line of thinking is that the collective enterprise of a school, production firm, 
symphony orchestra, or other organization possesses qualities that are greater than the aggregate 
of individuals within it, and that there is some identifiable “learning” that can be associated with 
the aggregate as a whole (Cook & Yanow, 1996). 

Some researchers have begun to explore what this might mean in the context of school reform, 
for example, by articulating the possible relationships between individual and organizational 
capacity building, as well as the ways in which the latter might be nurtured by policy actions 
(Goertz, Floden, & O’Day, 1995). The frameworks and findings of this line of research point to 
the importance of “communities of practice” within the school, which can both enhance or 
inhibit teachers’ ability to act on their personal knowledge, beliefs, and skills in their daily work. 
Subsumed within the concept of schools’ and school districts’ organizational capacity are various 
dimensions: vision and leadership, collective commitment and supportive cultural norms, 
knowledge or at least access to it, resources, and structures that are conducive to learning (Goertz 
et al., 1995). These dimensions define the organization’s receptivity to reform ideas and ability 
to sustain them over time. Others have focused attention on a similar set of attributes of high- 
performing school organizations and argue that they constitute a prerequisite condition for 
ambitious curricular reforms to take hold (see argument in Odden, 1995). Parallel work has 
invoked the concept of social capital (Coleman, 1990) to capture the “social relations and 
structures that seem to affect the capabilities for individual and collective action” in response to 
systemic or other reforms (Ball & Cohen, 1995, p. 7). This line of thinking goes beyond the 
boundaries of the organization itself to consider how societal relations and structures might 
determine, for example, what parents and others know of mathematics or other subject areas and 
how this “social capital” might influence the prospects for reforms within classrooms. 

What Learning Perspectives Explain 

By conceiving of systemic reform implementation and impact in terms of professionals’ 
learning, this perspective draws attention to the way systemic policies, the policymaking 
coalitions that fashion them, and the conditions surrounding these policies’ implementation teach 
the learners, be they teachers at the ground level, intermediaries, or the public “consumers” of 
the system’s “product.” Generally, the research on the reforms in California and other states 
argues that the “pedagogy” of systemic reform policies to date is more like the conventional 
approaches to instruction that reformers hope to replace in classrooms than the alternative 
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approaches that reformers advocate (Cohen & Barnes, 1993). Viewed in these terms, the failures 
of systemic reforms to date can be attributed to poor policy "pedagogy" (i.e., that policies 
communicate reform intentions in a didactic, monolithic way), limited opportunities for learning 
(i.e., that policies do not stimulate or support a variety of occasions and means for deeper 
learning about subject matter, instructional technique, and change strategies), and impoverished 
resources to support professional and organizational learning over time.* The evidence of greater 
learning in high-capacity settings can thus be a manifestation of greater attention by actors within 
these settings to the nature of the learning opportunities they construct (e.g., through professional 
development and the development of "communities of practice"). 

What Learning Perspectives Do Not Explain 

While this third perspective takes us deeper into the territory that systemic reforms are 
attempting to traverse, and offers a number of cautionary examples of the struggles individuals 
and groups may engage in while responding to such reforms, the perspective is inarticulate on 
several scores. First, work drawing on this perspective doesn’t yet help to see how learning and 
change may differ, and when it is appropriate that they be different (Cook & Yanow, 1996). In 
this sense, learning what systemic reforms mean and how to realize them in practice may not be 
the same as motivating and sustaining their actual realization in classrooms over the long term. 
Second, at the organization level, capacity to put systemic reforms into practice is not the same 
as actual day-by-day translation of these ideas into lesson plans, student learning activities, and 
performance results. What in the incentive structure, environmental pressures, or collective sense 
of direction moves schools to act on what they may have learned how to do? 



Conclusions 

It is time to take stock of what we know and don’t know about what intervenes between systemic 
reforms and mathematics and science classrooms. Despite the early stage in most NSF-sponsored 
systemic reform efforts, we probably know more about their likely prospects at the classroom 
level than may at first be apparent. Drawing on all of the sources reviewed above, some patterns 
seem clearly established and others highly likely. Based on these patterns, there are some 
important implications for those who wish to understand them better through further study and, 
ultimately, for those whose goal is to promote such policies. 



* It is easy to view state government's pedagogy of policy as monolithically traditional. There is 
some evidence from California, however, of educators being introduced to reform ideas in 
imaginative and nondidactic ways (S. Wilson, personal communication). 

What Systemic Reforms in Mathematics and Science Education Appear to Be Accomplishing 

The emerging evidence appears to support the following claims. 
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Systemic reform initiatives are hastening the diffusion of attractive , if challenging, , ideas about 
subject matter and pedagogy . Systemic initiatives have capitalized on their visibility, broad 
conception of target, and breadth of participation to infuse a particular set of reform ideas into 
professional conversations at many levels, as well as simply validating conversations about these 
ideas where they already existed. Whether or not these reforms can hold professional and public 
attention long enough to become considered “part of the way things are” remains to be seen 
(although reform ideas such as those embodied in the National Council of Teachers of 
Mathematics [NCTM] Standards have demonstrated remarkable staying power). 

Systemic reforms are mobilizing commitment , resources , and actors on an unusual , in some 
instances unprecedented , scale. Here, systemic initiatives appear to have scored some of their 
biggest "successes," although proximate ones. Unusual coalitions have formed; resources have 
been leveraged; and a variety of actors, including many teachers, are actively pursuing their own 
interpretation of systemic reform goals. The twin imperatives of broad participation and 
collaboration have been widely communicated, and diverse groups are heeding the call. 
Mobilization begets a diversity of interpretations, but that is not necessarily unhealthy. Over the 
long term, however, it is hard to see how the complex coalitions now forming or in place will 
stay aligned around the banner of systemic reform in these curricular areas. 

Systemic reform initiatives have made a modest contribution to the rationalization of regulation 
and resource allocation at the top of the system. By providing compelling rationales for 
connecting policies to one another and to widely held, if vague, goals, systemic strategies have 
for the moment made inroads into the superstructure of regulations, incentives, and other policy 
features at their source, that is, in a number of states and localities. This degree of coherence 
among policy elements does not penetrate too deeply, as yet, into the respective systems; 
furthermore, the intermediaries between the top and bottom of the system appear too variable 
(e.g., in capacity and will) to maintain and support the alignment of policy elements. Hence, 
long-term alignment around a single vision of excellent teaching and learning in schools seems 
unlikely. 

Systemic reform messages are reaching teachers in large numbers but are not necessarily 
transmitted with faithfulness to the original intention , nor are these messages likely to be fully 
understood by most teachers. The evidence to date, although from a small number of cases, 
suggests that teachers see quite different things in the same set of messages. Many teachers are 
grafting pieces of the reform vision onto an existing repertoire, with varying degrees of 
adjustment in the basic premises on which they view and utilize that repertoire. Understandably, 
instruction often resembles what has happened in the past, although with some superficial 
adjustments. 

Systemic reforms have stimulated a great deal of new learning among many individuals 
throughout the system. To date, there is more evidence of individual professional learning than 
organizational learning, however. And the learning that results is not necessarily what reforms 
intend, nor is the “teaching” consistently conducive to the kinds of learning most implied by the 
reform visions. 
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What We Don’t Yet Understand Well Enough 



These patterns and possibilities remain to be confirmed as the policies in questions endure the 
test of time. Existing efforts to monitor and assess current systemic reforms will be quite helpful 
in this regard. But there are specific facets of the response to systemic reforms that deserve 
closer attention over the years, for in them may reside clues to more effective and lasting 
promotion of these reform aspirations. 

For one thing, we need to understand better how reform messages move along the different 
avenues of influence available to policymakers. The nature of the transactions between reform 
promoters and the various intermediaries, as well as between the intermediaries and teachers, is 
not yet as clear as it should be. In addition, we need to know in greater detail how different 
avenues of influence reinforce each other (or not) over time — for example, how teachers’ 
“consumption” of reform ideas is related to local coalition formation. Or, research on the 
“conduits” through which reform ideas reach teachers would help to illuminate how ideas travel 
and evolve in transit. 

For another, we need to document how organizational and individual capacities grow and how 
the two are related. Where and how does individual learning about challenging science 
instruction, for example, get deposited in organizational routines? How does a culture supportive 
of science instruction take root in a school and sustain itself beyond the original cast of 
characters who helped set in motion a shift in organizational culture? 

Third, we need to understand better the transfer between learning what systemic reforms mean or 
how to realize them and the act of translating learning into classroom practice. 

Understanding these things better would offer insights into productive strategies — and the 
strategic limits of policy — to move the next generation of mathematics and science teaching 
closer to the vision that has motivated the systemic reform movement so far. 

Making Sense of the Power and Limits of Systemic Reforms at the Classroom Level 

Clearly there is much more to learn and know about the way this class of policy strategies works 
and how it does or might stimulate and support new forms of instruction in classrooms. While it 
is too soon to make confident pronouncements about the worth of these initiatives at the 
classroom level, there are some observations one can make about the means by which judgments 
may be made about their worth. Clearly, more than systematic evidence across time and a variety 
of settings will be involved. At a minimum, questions of the criteria for success and the public 
politics of reform will be part of the story. 



Sorting Among Standards of Policy " Success " 

Whatever evidence accumulates over the long term, the ultimate sense to be made of systemic 
reforms in mathematics and science education rests, in part, on beliefs about what constitutes 
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"success." Consider at least the following four standards of success at the classroom level that 
might be applied, both individually and collectively, to classrooms across the system: 

• Full embodiment of reform visions. This highest standard of success has clearly not been met 
yet by many, if any, teachers studied closely and, for reasons that are probably obvious, is 
unlikely ever to be met on a mass scale, at least not in the near future. 

• Widespread grafting of reform ideas onto familiar practices. This standard is easier to meet, 
for many reasons, and evidence is accumulating that many teachers are beginning to do just 
this. The standard begs a question, however, about how much grafting indicates success, and 
whether the shape of practice must ultimately change in some degree or whether old and 
new, potentially contradictory ideas can coexist, and how, in a "successfully" reformed 
classroom. 

• Widespread professional learning among classroom teachers. Short of the higher standards, 
one might declare that systemic reform policies are worth the investment if large numbers of 
teachers raise questions about their own practice, consider new possibilities, and engage in 
various modes of professional learning, whether or not there were evidence of any change in 
current practice. The presumption would be that professional learning accumulates and will 
ultimately nudge practice in the desired direction. 

• Incremental increase in "system averages. "* Even more modest might be a standard of 
success that, by gradual stages, countable indicators across the system (e.g., the proportion of 
teachers using manipulatives in elementary mathematics teaching, the proportion of 
elementary teachers attempting science instruction with hands-on learning activities) would 
reveal trends in the directions assumed by reform. To be sure, these indicators by themselves 
may indicate little about the depth to which reform ideas have penetrated, but they may 
indicate a gradually developing predisposition toward change. 

All of these standards are invoked in current discussion, and it depends in part on whom one 
listens to. Ultimately, it probably matters most what teachers themselves internalize as the 
operative standard to which they hold themselves accountable. That many teachers are willing to 
consider new possibilities for their teaching and hold themselves to a higher standard seems 
likely, given the evidence that has accumulated so far; their capacity to sustain that commitment 
over the long term remains uncertain and depends in large measure on how supportive the 
surrounding context will be. 



* lam indebted to A. Zucker, SRI International, for suggesting this idea. 

Public Perceptions and Patience 

A key part of the surrounding context will be the degree to which the public perceives good 
things to be happening in mathematics and science classrooms and is willing to give the reforms 
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time to demonstrate how far reaching the resulting improvements might be. To the extent that 
the public likes what it hears and sees about the process or products of reform, then it is likely to 
tolerate-or even support with varying degrees of enthusiasm-the slow, complex attempt to make 
the educational system demand and support more challenging instruction in mathematics and 
science. But public patience is in short supply these days. Systemic reforms have much to do to 
portray their policy initiatives' impact on classrooms in a light that will engender and sustain 
public confidence. As they seek to offer such portrayals, reformers compete in an active 
marketplace of ideas and concerns. To their credit, systemic reform strategists appear to 
understand the importance of a visible presence in this arena. The outcome of their efforts in this 
busy arena, however, remains uncertain. 
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